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ABSTRACT 

fM aee u „ n yCar at the Greater Lawrence Technical School 

Uto achusetts), students are identified who lack the necessary basic 
skills to successfully complete a typical high school program. This 
problem led to the development, of an alternative instructional 
program designed to accomplish remediation in the basic skills using 
computer-based instruction. The Greater Lawrence program is designed 
to promote and foster individual, independent growth and skill 
accomplishment for secondary students in the skills of mathematics 
and reading. The purpose of this study was to evaluate and review the 
appropriateness of the reading and math program in terms of its 
impact on the reading or math scores for those students assigned to 
the developmental skills lab. The study was conducted to determine 
the extent to which the degree of use of the computer-based 

ac h ilvt T 1 Pr ° gram affeCted gr ° UPS ° f Stud6nts * n standardized 
achievement measures. Progress for each student was calculated on a 
pre post test gain basis. The mean normal curve equivalent gains on 
reading comprehension and mathematics from pretest to posttest was 

Provide 1 - Sign i fiCa !! t ' ThiS alternati ^ supplementary approach to 
provide reading and mathematics instruction at the secondary level 
demonstrates improvement in relevant test scores on standardized 
achievement tests. Particularly positive were the mathematics 
achievement gains. This report provides the results of the program's 
evaluation. (Contains over 50 references.) (Author/MDH) 
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ABSTRACT 



Each year at the Greater Lawrence Technical School, students are 
identified who lack the necessary basic skills to successfully complete a 
typical high school program of studies, Such students consistently fail 
to meet agreed upon standards as measured by state mandated basic skills 
examinations or other standardized tests of achievement in their school 
subjects. This persistent problem led to the development of an 
alternative instructional program designed to accomplish remediation in 
the basic skills using computer-based instruction. This approach to 
basic skills remediation uses the power of computer technology to provide 
instruction, tutorial help, and drill and practice on the basic skills in 
math and reading. The Greater Lawrence program provides an intensive 
approach with one-to-one attention in developing each student's program 
of learning. Other common features include consistent reinforcement and 
feedback about performance for both the teacher and the student. 

The Greater Lawrence program is designed to promote and foster 
individual, independent growth and skill accomplishment for secondary 
students. The school's efforts are aimed at enhancing student 
achievement in the skills of reading and mathematics. This evaluation 
and review examined the reading and math program at the Greater Lawrence 
Technical School. The purpose of the study was to assess the 
appropriateness of the program in terms of its impact on the reading or 
math scores for those students assigned to the developmental skills lab. 
The study was conducted to determine the extent to which the degree of 
use of the computer-based instructional program affected groups of 
students on standardized achievement measures. 

The IDEAL curriculum and computer system is a computer-based 
instructional system used to reinforce skills in reading and 
mathematics. The evaluation reported here provides information regarding 
its effectiveness at the secondary level. The general design of the 
evaluation was to compare students on achievement gains using school year 
1987-88 standardized test results. Study participants included Chapter I 
teachers and other faculty and students in grades 9 and 10. To measure 
academic progress students took standardized pre- and posttests in 
reading or mathematics. The Stanford- TASK is a standardized achievement 
instrument that is used for this purpose covering objectives for both 
reading (TASK-R) and mathematics (TASK-M). This report reviews the 
instructional impact of the program and procedures for data analysis are 
described along with results. 

The present study made use of achievement scores from the 1987-1988 
school year and traced the progress of secondary students of two groups 
receiving either remedial mathematics or reading instruction. The school 
employs a four level tracking system to group students of different 
ability levels. Generally, students in track four (lowest ability) are 
assigned by class to the computer instruction lab based on test scores on 
the Differential Aptitude Test (DAT); many are also assigned to Chapter I 
remedial instruction programs for developmental reading and math. 



i i i 



9 

ERIC 



4 



Progress for each student was calculated on a pre-post test gain 
basis. The mean normal curve equivalent (NCE) gain on reading 
comprehension from pretest to posttest was statistically significant. 
The mathematics gain from pretest to posttest was also statistically 
significant. These gains exceeded expectation by more than a quarter of 
a standard deviation unit and, therefore, may also be considered as 
"educationally significant." Results showed significant gains for 
students in math and reading. 

This alternative supplementary approach to provide reading and 
mathematics instruction at the secondary level demonstrates improvement 
in relevant test scores on standardized achievement tests. Particularly 
positive were the mathematics achievement gains. This report provides 
the results of the program's evaluation (over three-quarters of a 
standard deviation unit). 

Computer-based instruction involves the student interacting with a 
computer along with teacher supervised instruction for reading or 
mathematics. The teachers reported favorably on the use of the computer 
as a motivating learning tool. The computer also provides an 
instructional management system that links the diagnostic items and 
teaching modules for use by the high school mathematics and reading 
laboratory so that each student is working with appropriate lessons. 
Teachers presented a diagnostic math test at the beginning of the year to 
ascertain entry levels for each student in the mathematics curriculum. 
Two emerging forces can help to provide a sound basis for the 
introduction of these teaching tools into the secondary curriculum: (a) 
inexpensive, cost-efficient processing power to provide the necessary 
instructional information for decision making; and, (b) growing 
understanding of the cognitive tasks required in the curriculum. This 
program at the secondary level has furtner provided evidence to encourage 
course development and demonstration. The computer-based instructional 
component assists teachers in reexamining the basic skills curriculum in 
light of changes brought about by the introduction of these new tools. 
This system contains instructional components of areas that educators 
traditionally consider important to the learning process. 
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EXECUTIVE SUMMARY 
EVALUATION OF READING and MATH PROGRAMS 



Evaluation Study of the Curriculum System 

The Greater Lawrence Technical School addressed student basic skills 
attainment in the computer-based instructional lab. The school began 
installation and implementation of this program at the beginning of the 
1987-1988 school year. This study reviews achievement of student skills 
through the implementation of a standardized testing program evaluating 
the basic skills remediation program with use of the accompanying 
management information system within the microcomputer network. The 
Merrimack Education Center conducted this comprehensive evaluation of the 
IDEAL Learning program in the setting of this secondary school. Adequate 
provisions were made to identify conditions of the study to put primary 
focus on student achievement variables. 

Evaluation Questions . The following were the main evaluation 
questions in this study: 

1. What affect does the formal instructional program IDEAL 
have on achievement scores of students at the end of one 
year of the program? for reading? for mathematics? 

2. To what extent does the intervention provide skill 
remediation for low achieving students wno would typically 
be identifed for basic skills remediation by the state test 
of basic skill s? 

3. Is supplementary instruction (with reinforcement drill and 
practice) related to the level of gains for classroom/ 
computer-aided instruction? 

In answering these major questions, the purposes of this report and 
the program described here are to: 

(a) understand what the data show about the performance of 
students in the reading and math areas; 
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(b) understand the program design and how computer-based 
instruction helps the achievement of students; 

(c) make inferences regarding what interventions are best 
undertaken with computer-based instruction that would yield 
the greatest benefit for some investment of resources. 

Because these microcomputer systems are relatively new, there yet 
exists little documented evidence of their effectiveness. School systems 
must make decisions on the basis of developer's claims rather than 
research or proven practice. These new basic skill programs, with goals 
and objectives in the major curriculum areas, must be carefully 
evaluated; and, these new curriculum strands must also be viewed in the 
larger context of the secondary school where their impact can be reviewed 
in relation to other approaches that have been used traditionally for 
these same purposes. "\ 

Therefore, the evaluators conducted the study to determine whether 
the instruction for reading and mathematics provided to track 4 students 
(lowest ability level) in the school results in a detectable effect on 
student academic growth. As part of a regular assessment program 
conducted by the Greater Lawrence Technical School, test scores were 
analyzed and were duly reported by the teachers to mark student 
progress. The students are compared to similar groups of students who 
took the test as reported by the publisher in the norms booklet (a 
norm-referenced model). Criterion-referenced test scores were gathered 
and recorded by the teachers using the a teacher made test of mathematics 
to determine a functional level for each student when entering the 
computer-based curriculum. Functional levels were also determined for 
reading when students were entered into the IDEAL 1 s curriculum on the 
microcomputer for their instructional lessons. 

In this study, a set of class test scores was examined from the 
school district's records and reviewed as they relate to the reading and 
mathematics program. Data reported in this evaluation study are from the 
developmental program featuring reading and mathematics at the secondary 
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level. The district compiled test scores for several subtests during the 
fall of 1987 that were used in the evaluation along with the posttests in 
spring of 1988. These data, compiled by the teachers and entered into 
the management information system in the lab, were used to perform 
computer analyses. 

Basically, the testing data are of two types: the percentile scores 
showing mastery for the reading or the math test is the first type of 
data collected. Using the school 's testing schedule, entry level data 
were compiled for each student in the program. The sampling of students 
used in this study for reading was small covering a portion of the 
student body at grade 9 who were assigned in track 4 over a school year 
but who were not assigned to Chapter I remedial instruction. The data 
set for mathematics was larger covering grades 9-10. Test scores 
collected and compiled were the total reading, total math, verbal, 
numerical ability, basic skills math, basic skills reading, and a 
teacher-designed criterion referenced test. Additional information on 
students in the remaining tracks (grouped by ability in tracks) was 
compiled only on the fall pretest. 

Literature Pertaining to Secondary Achievement 

Factors related to achievement have been extensively studied and 
reported in the research literature. MEC has extensively studied basic 
skills intervention programs for more than a decade and performed a 
thorough review of the topic for this study with the purpose of setting a 
context of variables, those considered as possible correlates of 
achievement. 

National reform literature and mandates for effective schools 
continue to emphasize that basic skills in reading and math need to be 
provided at the secondary level for students of all abilities. As early 
as 1979, the Massachusetts Department of Education undertook an objective 
statewide assessment to determine if students are mastering basic reading 
and math skills. While achievement at the elementary grades is 
consistently rising, nationwide, older students do not necessarily 
develop the higher level skills without a program aimed at these higher 



order tasks. Assessment statewide is accomplished through the 
implementation of a testing program intended to provide an objective 
overview of the extent and nature of student achievement in these 
critical skill areas and content areas of reading and .the practical 
applications of math skill. 

Computer-aided systems to remediate basic skills have been available 
for over a decade. Evaluation studies have shown impressive results with 
consistent achievement test gains that are sustained. These computer 
managed programs go beyond the delivery of educational programs through a 
regular or traditional classroom dependent upon paper and pencil 
instruction. 

In the past, all large computer systems required the power of a 
mini -computer such as the Data General Nova IV or other mainframe for 
adequate storage space and processing capacity. Advances in technology 
have recently led to the development of less costly, micro-based basic 
skills remediation products such as the one recently installed in the 
Greater Lawrence Technical School. These microcomputers have more 
capability than the earlier school machines when effectively used in a 
networking arrangement offering easy access to large hard disc storage 
devices (as opposed to floppy discs that must be constantly re-cycled and 
re-entered into the microcomputer). These advances in the power of 
technology have allowed the developers to produce basic skills 
remediation packages that are comparatively within the financial reach of 
most typical public schools in Massachusetts. 

There are several factors that are_implicit in the computer-based 
instructional process. The review and discussion of this project with 
teachers determined that the curriculum is related to areas that 
educators traditionally consider important to the learning process. 
Like their larger predecessors (minis and maxis), these microcomputer 
systems with hard disc "file server" offer a variety of curriculum in the 
key areas such as mathematics, reading, and communication skills that can 
be focussed on the remedial programs of individual students. 
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A number of studies have described factors such as general 
intelligence level that is apparently related to students 1 school 
achievements. No IQ scores were available for these students; however, 
we were able to use the DAT-VR (verbal) a stand-in for IQ because it is 
well-known to correlate to "g" factor of intell igence. The Task-M and 
Task-R were used to measure achievement in mathematics and reading 
respectively. They were readministered in the spring at the end of the 
school year and this provides a measure to compare the student's DA T VR 
and numerical ability in the fall battery used to estimate 
"ability/capacity" for each student when assigned to a track. 

Students 1 attitudes towards school and success are variables 
recognized to likely influence level of achievement in reading and math. 
Also, student behavior ratings as assigned by teachers have a high 
relationship to achievement and are often significant when assigning 
pupils to tracks. % DAT scores at the Greater Lawrence Technical School 
were used to assign students to tracks, have high ratings indicating 
self-reliance and cooperati veness along with other motivational variables 
assigned to higher tracks. These are also predictors of intelligence and 
aspects of achievement. In many studies such as this one, it has been 
found that a critical variable is the teacher's belief that the students 
are the most critical agents in their own learning. This is reinforced 
by their selection of independent computer-based activities for their 
curriculum program at the secondary level. The teachers' degree of 
belief in the importance of independent learning and studying has been 
found to contribute substantially to the differentiation of assignments 
to different classes. The recommendations of the teachers are highly 
correlated with the teacher's belief that the student will do well. The 
lower expectations of the teachers for a portion of the student body may 
become the Rosenthal -Jacobson self-fulfilling prophecy. Learning process 
variables that are important at the secondary level include student 
attitudes toward school and continuing education; teacher attitudes and 
respect for students as learners; and, student and teacher beliefs that 
learning is occurring and can succeed. 

Another variable, which is an aspect of "locus of control", related 
to differences in performance has been reported in the literature as "the 



degree to which the student felt that s/he was responsible for his/her 
own successes and failures in school." The provision of appropriate 
kinds of experiences, student support and guiding by their teachers, has 
been shown to be enhanced with adequate resources. Several other 
variables have been identified in the literature as differentiating 
various pairs of comparison groups. 

There may be common factors that exist which can help to explain why 
some students are doing better over and above the instructional program. 
For example, do aspirations of parents provide a more supportive 
background despite their student's innate capacity? Do their teachers 
hold higher expectations for them or make special efforts to overcome 
some of the learning deficiencies? Do the students assigned to higher 
tracks than track 4 have certain traits or personality characteristics, 
or are they merely more or less "intelligent"? The answer is a complex 
mixture of factors and these factors should be further studied as the 
school considers programs or other school supportive services. 

In national research studies, there is a general trend for 
achievement scores to be higher as the frequency of reported recreational 
reading (books read) increases and as the parental reinforcement of 
reading occurs in the home. This is indicative of the "practice makes 
perfect" rule. For this reason, we reviewed the data collected on the 
Massachusetts Assessment of Basic Skills. 

Because of its ability to record and represent process (e.g., 
mathematics problems, etc.) the computer provides a powerful , motivating 
tool for focusing the student's attention directly on tne steps or 
process of thought in working out a math problem. It is possible to 
capture the processes by which the student performs the work: 

This process trace or audit trail can become a useful 
object of study for students trying to improve 
performance. Reflection is important to learning because 
it is possible to reconfigure a process representation so 
that students can see separate aspects of <*4he process 
together and can view the process itself from perspectives 
not .seen before, (Bolt, Beranek Newman) 

6 - 
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Impact and Implications , Of significant interest in this study were 
the procedures for curriculum management, staff training, and 
student/teacher feedback provisions. Teachers considered several factors 
when setting up the lao as they explored concepts crucial to development 
of this new computer-based learning environment. Teachers were 
instructed to focus on the process and the product and the computer's 
ability to record the students 1 demonstrated skills and academic 
achievement. Cognitive and pedagogical issues were considered relevant 
to the learning environment. While working with the curriculum and 
devising an entry level test, the math teachers focused on issues and 
concerns identified on the impact of this curriculum system for their 
general education program and the specific skills to be taught. Faculty 
worked with Merrimack Education Center s^aff to determine the 
implications of this computer-system on their approaches to learning and 
teaching together with ways in which to accommodate the changes in 
educational practice required that would result from the application of a 
"new generation" of tools using technology. Teachers worked through 
inservice education to determine now they would adapt the approach to 
specific learning and specific learners. (Figure 1 depicts the lab 
set-up. ) 
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Evaluation Design Model 



In this evaluation of the IDEAL implementation at Greater Lawrence, a 
pretest - posttest, norm-referenced design was employed. Because all 
students of a similar ability use the IDEAL computer-based program, it is 
difficult to locate appropriate control or comparison groups as it is not 
always wise to establish a "waiting list"; thus, comparisons are made 
with the publisher's established norms. Those students participating in 
the current computer-aided program were chosen according to the same 
standard criteria (and testing data) that are generally used in secondary 
schools to establish this type of remedial class or for identifying 
students in need of remediation. 

For this current cohort of students the study reports test results 
for the Stanford TASK-M and TASK-R to determine student progress from 
pretest to posttest. The normal curve equivalent is the statistic used 
for this comparison and this NCE score (on a standard Z scale) is used to 
compare with the data available from the publisher on the norming 
population. 

The model selected allowed direct comparison of net learning gains 
within this particular year. The statistical techniques of analysis of 
variance (IBM-PCANOVA) to assess the homogeneity of groups and analysis 
of DAT-Im were used to examine abil ity /capacity levels of incoming 
students in grade 9. All data will be maintained in a computer data base 
so that future studies, of a longitudinal nature involving greater 
numbers of students in several schools, can be prepared in future school 
year comparison. 

The design for this evaluation is snown in Figure 2*, Data analysis 
were prepared to compare trends in these data for students attending the 
Greater Lawrence Technical School and to draw direct implications for 
this type of remedial, program in reading and mathematics. 



*This is Model A as developed by RMC Corporation for the U.S, Office of 
Education, 



Figure 2 
Data Collection Design 
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Study Activities 

The present study made use of achievement scores from 1987-1988 
school year. The study described in this report traced the progress of 
9th and 10th grade students of two groups receiving either remedial 
mathematics or reading instruction. Generally, students in track four 
are assigned to the computer instruction lab for remedial math 
instruction programs or developmental reading and math. in specific 
terms, this study attempted to: 

(a) follow the progress of students in grades nine and ten with 
test scores at the beginning and end of the school year; 

(b) describe the levels of achievement in reading and 
mathematics; 

(c) identify any subtests that might differentiate between the 
groups of students who received the computer-based 
instruction (e.g., math, reading, basic skills, numerical 
ability, etc.). 
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The students in the lab received tests as part of their achievement 
testing at the end of the school year, making it possible to identify a 
variety of subgroups such as students who remained in either the high or 
low scoring group on the tests as they attended the high school classes. 
The instruments used to gather quantitative data for the evaluation of 
this : program are reliable and valid. Use of the Stanford (Task-M and 
Task-R) achievement test, with common norms across several years, permits 
comparison of students in this school with other typical secondary 
students who took the Stanford as part of the sample for norming the 
test. The normal curve equivalent (NCE) is the score used to make this 
comparison over the testing periods.* 

Findings from this study comparing groups of readers and math 
students assigned to different instruction using the computer lessons are 
reported here and we have offered some suggestions concerning factors 
that might account for a group's differential performance. Variables on 
which the groups were measured included reading and math achievement, 
number of skills mastered in the criterion-referenced (levels) test. 

Basically, the evaluation model looked at the pretest and posttest 
mastery scores for the reading and math subtests. Using the DAT entry 
level scores are collected on each student in the junior high school in 
the spring of each school year. The Stanford TASK achievement test is 
administered at the beginning of the freshman year in the Greater 
Lawrence Technical School and again at the end of the school year; this 
provides a measure of total reading and math (TASK-R and TASK-M). 
Meanwhile the DAT, given only once at pretest, provides a score for 
numerical ability and verbal reasoning. 

Data Collection and Analysis 

As part of the school's assessment program to determine the 
effectiveness of instruction, test scores were compiled on standardized 
achievement measures. The Merrimack Education Center reviewed and 
analyzed these test scores for students who had remained in the reading 
program or math group for the full year. The outside evaluators reviewed 
the data and verified descriptive and inferential statistics. The 
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outcome measures wera the reading achievement test and the math subtest 
appropriate for this evaluation. The comparison of test scores from fall 
to spring for the student participants in grades 9 and 10 were analyzed 
from tests administered over the school year that were indicative of 
program impact. Stanford TASK Levels I and II A were used. The 
evaluation procedure for this report is summarized as follows: 

1. Standardized achievement test data were reviewed; t-tests 
for correlated data (matched pre and posttest scores) were 
used to determine gains. 

2. T-tests for correlated data were used to determine gain in 
the subtests (i.e., total reading, math). Data analysis 
was conducted in accordance with the norm-referenced Model 
A, as promulgated in the national Chapter I evaluation 
effort of recent years. 

3. The matched pair students t-test is equivalent to a one-way 
ANOVA (analysis of variance) and was used to determine it 
gains were significant. 

Because the students were all pre-selected for tracks in the 
vocational program, it was impossible to do a "tight" research study with 
control groups. The two instructional groups (reading/math) differed 
markedly from the general student population in that their OAT test 
scores were lower to begin with. These students could be expected to 
perform very differently on a standardized reading or math test given at 
the end of the 8th grade. It was hoped that, by monitoring the progress 
of several of these groups over the 9th or 10th grade year and examining 
aspects of their performance, some clues about performance and 
levels of reading/math achievement might be found. The study results are 
viewed to determine the: 

1. pattern of achievement gains by subject area 

2. relative magnitude of gains expressed in NCEs 

Data Collection Sources 

Students are given a standardized achievement test to determine 
students 1 current math and reading levels. The Stanford TASK achievement 
test is a widely used instrument with high validity and reliability that 
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measures relatively important educational goals for both reading and 
math. All scaled scores and raw scores were converted using the norms 
booklet and the norm referenced model employing national norms as a 
comparison for this report.* Each of the skills of the math levels 
program was also examined as to the increase or decrease in the 
individual student's test scores over the time span of one school year 
when students took a teacher made test for IDEAL Levels. 

The individual curriculum components for reading and mathematics are 
analyzed on NCE or percentile score gains and they are analyzed for 
significance using difference of means tests (t-test) for matched pairs. 
Individual test items cumulate to make a composite score of Reading and 
Math on the Stanford TASK. Thus, the achievement scores represent 
several different curriculum objectives; among these are: 

1. Comprehension of what is read 

2. Vocabulary development 

3. Mathematics computation 

4. Mathematics problem solving 

Teachers have examined the tests and the curriculum to review their 
stated curriculum objectives. Teachers indicate that comprehension 
receives major emphasis in the reading program. Teachers express 
satisfaction with student achievement on the reading objectives, and 
overall performance on achievement tests. A teacher-made test was 
designed as a diagnostic measure to determine entry level placement for 
students in the computer instruction math lessons; a math levels test 
(correlated to their achievement of objectives) was administered in the 
spring of the school year to determine growth in the curriculum. (See 
Appendix. ) 

The population of students tested was large because the DAT is 
initially given to all freshman students from all feeder schools. The 
average scores of the 9th grade (entering freshmen) were at the 27th 
•percentile in math, indicating that the students are scoring lower than 
the typical student in a general education program. 

- 13 - 
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The Normal Curve Equivalent is the standardized score (z-score) used 
for the analyses in this report.* The norm referenced design assumes 
that pupils will tend to hold their percentile rank from year to year 
unless their curriculum is modified in an instructional program. 
Therefore, an increase in rank on the percentile scale, reflected on the 
NCE scale, is evidence of the benefits of an instructional program. For 
example, if students score at the 20th percentile in the fall, then their 
spring score will be compared to the score equivalent of 20th percentile 
students on the spring norms. Assessment of the program's impact can 
then be interpreted on the percentile scale and the NCE scale. NCE's are 
normal curve equivalents; this NCE scale closely matches the percentile 
distribution, hence has some of the qualities of interpretation inherent 
in percentile scores. However, the scale is also an equal interval scale 
and can be used in arithmetic calculations, while percentiles cannot. 
The NCE scale permits percentile scores from any norm-referenced test to 
be transformed to an NCE equivalent score. Scores can then be aggregated 
across grades 9 and 10 to determine the overall impact of the program. 
Whenever the NCE gain is greater than zero it means that the students 
profited from participating in the program. 



*NCE's are Normalized Standard Scores with a mean of 50 and a standard 
deviation of 21.06. The NCE scale is an equal interval scale unlike the 
percentile scale and so they can legitimately be aggregated and 
averaged. NCE scores are generally considered to provide the most 
comparative information in equal units of measurement. 



RESULTS OF ANALYSES 



The outside evaluators reviewed the achievement test results and 
verified scores on tests as sources of data for this report. Outcome 
measures included the Stanford TASK R and TASK M reading and math. Using 
the test data, the evaluators were able to compare measures of reading 
and math on the standardized test administered. Data compiled at the 
school were used for analyses performed in this report. 

The test results indicate that a typical student in this school 
scores below the national average (50th percentile) in reading and math. 
The average overall achievement, as indicated by the test, is 
approximately 25th percentile for both reading and math. 

In the present study, groups of test scores were compiled according 
to the instructional groupings of the school for the reading or 
mathematics program. These students were identified as fair or poor in 
their reading and math skill development according to test scores and 
their assignment to track four. Tests were given at the end of their 
eighth grade year and students consequently assigned a track; then the 
lowest performing students are assigned to either the reading program or 
the math program or both. These students * were followed throughout the 
year by their teachers and then some additional posttesting was 
completed, in the hope of finding common areas that would indicate on the 
subtests any differences in reading or math achievement for these pupils 
assigned to different classes. 

It was expected that the results of these comparisons would on the 
whole be similar, because many of the same students remained in track 
four and were administered both pre and posttests. It was thought, 
however, that differences in patterns of test scores might emerge for 
grades 9 and 10 analyses because of the greater general maturity of the 
students as well as the instructional program. (Degree of independent 
learning success is thought to be developmental). Also, it was hoped 
that the instructional treatment would be a predicting factor for greater 
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achievement when the student had been assigned to the developmental 
skills classes using the microcomputer. 

In order to determine whether significant gains were made on total 
scores, the pretest and posttest (fall to spring) NCE scores were 
examined on the subtests. A preliminary run on testing data for all 
students in the total population we, prepared (N= 1 55) • Descriptive 
statistics on the DAT (basic descriptive information) including means, 
medians, .and standard deviations were obtained for tne total group of 
students and for the sub-groupings (track 3/4 assigned students). 

The general student body had higher capacity to begin with and this 
must be taken into account as the data are analyzed. For example, the 
average or typical student scored as much as 10 percentile points higher 
(on the percentile scale) than the group assigned to reading and as much 
as 15 %i 1 e points higher on the math test. The students in the 
computer-based instructional program are indeed in need of remediation 
and need the extra supplementary assistance provided by the computer. 

All analyses herein described involved the use of Normal Curve 
Equivalent Scores (NCE's)*. At each measuring point, each student's NCE 
was recorded. For the presentation of tables, these NCE's have also been 
converted into percentile rank. These conversions were performed by 
table lookup procedures. Only the NCE scores were used for statistical 
analyses because they alone allow near complete flexibility in assembling 
groups for analysis. 



*If a student does gain on the NCE scale, it can be stated that: s/he 
gained more than one year's growth in the school year; and, s/he improved 
her/his ranking among peers relatively in an upward direction. 
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READING 



Reading items on vocabulary and comprehension are gathered into one 
composite score for reading on the Stanford TASK-R subtest. All students 
taking the reading program were gathered for analyses when both pretest 
and posttest scores were available (N* 24). Data for student achievement 
on reading are shown in Table 1 for 9th grade students assigned to the 
reading lab. Achievement tests (Stanford TASK) are designed to give 
evidence of how much a particular student has learned. They are designed 
and given for the purpose of rank ordering students in comparative 
achievement. 

Evidence of the impact of the school's program, through data 
collected on reading, is displayed showing that gains were made on the 
reading test administered in the spring of grade 9, after students 
completed the year's program of studies. These data represent pre- and 
posttest scores on the same students at the beginning and at the end of 
grade 9. Means and standard deviations were computed for the students 
attending the lab and the differences of the test scores from pre to post 
(fall to spring) on the reading subtest were then assessed by means of a 
t-test. The test publisher's norms were the reference for "expected 
end-of-year reading performance" and this was compared witn "actual end 
of year performance" on the percentile scale and the NICE scale. 



Table 1 

READING Stanford TASK-R 
All Students with both Pretest and Posttest Results - Grade 9 

Fall Grade 9 to Spring Grade 9 



Students in reading Lab 
N~ 24 



Pretest Posttest Gain* 



13% 



20% +7% points 



9 
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Assessment of the impact can be gained from examining the percentile 
scores in Table 1 or the NCE scores in Table 2. Percentiles are used in 
norming tests to display the scores of all persons taking the tests 
during the period of standardizing the instrument. This allows the 
reader to examine the results of achievement tests according to whether 
or not they correspond with the large numbers of persons who took the 
test in the developmental stages and to draw concl usions related to 
achievement. Favorable results were found for the students in reading 
and this can be readily seen by examing the gain column. In other words, 
the student growth rate was faster than these same students had 
previously demonstrated. In this case, the students as a group would be 
considered to have performed better than expected. From examining Table 
1 it can be seen that there is evidence that the support offered by this 
secondary school supplements the students instructional program and is 
effective in accelerating the student's progress, thus compensating for 
his/her previous low achievement level. 

Total reading for the students completing developmental reading at 
the Greater Lawrence Technical School showed an overall gain for this 
composit^of the subskills in reading of seven percentile points (+7%). 
An increasing percentile rank from pretest to posttest is indicative of a 
faster growth rate than students had previously demonstrated. Without 
this supplemental instruction, these students would be expected to remain 
at the same level with respect to the test publisher's norm group. 

NCEs are derived from the percentile scale in look-up charts supplied 
by the publisher. The NCE scale is a relatively "new 11 scale developed 
and disseminated by the U.S. Office of Education. According to OE, "if 
the average relative performance of the students in the spring testing is 
better than their average relative performance in the fall testing, then 
the percentile of the average posttest achievement score would be at a 
hi gher 1 evel than the percen ti 1 e of the average pretest achi evemen t 
score. Thus, for students who are participants in the computer-based 
instructional lab, the norm-referenced model presumes that a 
better-than-expected performance is attributable to the effects of the 
instructional program. Within this framework the treatment effect 
attributed to the lab program is the observed post test performance minus 
the expected no- treatment post test performance. The norm-referenced 
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model is presented in terms of achievement scores that have been 
transformed from raw scores to percentile equivalents and then to NCE 
scores. 

Generally, the typical student in the remedial lab reduced the number 
of incorrect items on the reading subtest. Table 2 indicates that a 
typical student in the reading program started out with a mean of 26 MCE 
at the beginning of the school year and scored 32 NCEs on the posttest in 
the spring at the end of grade 9. For all students in the data file for 
grade 9 who attended reading classes and used the microcomputer, the mean 
NCE rose +6 points on the NCE scale. For example, the typical student in 
the developmental reading program moved ahead in rank order six places 
from pretest to posttest. 

Therefore, in this sample the students assigned to reading outpaced 
the students in the norming population test score when they had completed 
grade 9 on the the computer-based instructional practice. It is 
especially significant noting that the reading students start out lower 
than the 25th percentile to begin with, and thus have a harder hurdle to 
improve their rank placement; many students (21% of the remedial reading 
students) by the end of the year make a quartile shift into the 2nd 
quartile from the first. Regardless of the fact that the students 
assigned to reading are generally of lower capability (DAT Battery* 24% 
for reading students as opposed to 39% for aVI_ students), these reading 
students showed a gain of +6 NCEs whereas the typical student in the 
norming population would be expected to remain at the same score or 
percentile level from fall to spring. 

A students t-test was conducted to determine if the gain proved to be 
significant with this statistic. The reading analysis was performed 
using the students t-test to evaluate statistical significance. Table 2 
presents the data for the students t-test. The pretest to posttest 
statistic for the reading students was 4.847 indicating this was highly 
significant especially for the small number of students attending reading 
(only twenty four with matched pre and post tests). The level of 



*DAT verbal score is used here to substitute for IQ because none was 
available and verbal ability is a good measure of "g" or general ability. 
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significance was calculated to be less than <.0005 interpreted as a 
difference that would not be obtained by chance alone in more than 2,000 
replications of the study. Students show gains in total reading on the 
NCE scale and their average gain (+6) approximates half the standard 
deviation of either testing (pre/post). For example, the gain of seven 
percentile points is greater than the standard deviation of 13 on the 
pretest and is also greater than the posttest standard deviation of 12. 
Gains of even one quarter of one standard deviation are considered 
"educationally significant." Also, the fact that the standard deviation 
dropped from 13 to 12 is also educationally important showing that the 
variability 1n the group was "tightened up" due to the instructional 
program. 

Thus, we can conclude that the student does gain and it can be stated 
that s/he: (a) gained more than one year of growth on the reading test; 
and, (b) improved his ranking among his peers by 6 steps (changed upward 
6 places). 



Table 2 




READING — Stanford TASK-R 


All Students with both Pretest and Posttest Results - Grade 9 


Fall Grade 9 




Data File; GR. LAW. READ 




Paired Samples... 




Variable: Column 4 


Column 5 


Mean: 26.027 
Std. Deviation: 13.366 


31.807 
11.993 


Paired Observations: 97 




t-statistfc: -4.847 
Degrees of Freedom: 96 
Significance: 0.000 


Hypothesis: 
Ho: |il = }i2 
Ha: ji 1 * jl2 
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According to the U.S. Office of Education, even a one point gain on 
the NCE scale* is significant and, for this size population (twenty-four 
students) the point gain is quite significant. Statistical analysis of 
t-test (difference of means test) between pretest and posttest NCE means 
on the total reading score was statistically significant, in a positive 
direction. Thus, the comparison is significant showing positive gains 
and the differences between the means (fall to spring) were significant 
on the subtest at the end of Year 1 . 

It can be concluded that program changes made with the installation 
and implementation of computer-based instruction using microcomputers 
with a file server (hard disc as opposed to floppy disc drive) have 
impacted on the scores resulting in the gain of +6 NC£s for the typical 
student who was assigned to the reading lab. Observable progress with 
students is recorded by classroom teachers as the students accomplish 
more of the reading objectives on the microcomputer. This is noted by 
the teacher-selected objectives and discussion of issues with 
the teacher as wel 1 . 

We can make the inference that the instructional program in reading 
is maintaining student skills and that more skills and objecti ves in the 
school's curriculum are added when additional supplementary instruction 
is offered through computer-based instructional lessons. Thus, the 
school's curriculum is having a differential effect on the students of 
varying capacity and on those students in tracks 3 or 4 assigned to the 
reading lab. In similar studies conducted at the secondary level using 
CAI, MEC has noted that students often take two years to show a reading 
gain when such a new program is initiated. Indeed, often students at the 
secondary level actually go down in their reading score from grade to 
grade. 

When we look at track four students (the lowest performing) assigned 
to reading classes, there is a significant gain of +6 NCEs or +7 
percentile points. Students get additional test items correct on the 
post test (compared with fall) and enough additional items to push them 



*Gain of +9 NCE points represents +11 score points; represents 

increase of 13% in terms of raw score points . 
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to a higher percentile. The students undoubtedly answered a few more 
items correctly on their 9th grade posttest than they did on the pretest, 
enough to increase their percentage score (and consequent NCE score). It 
takes about seven or eight more correct answers to raise their score 1 or 
2 NCEs. 

By definition and the criterion of selection, students assigned to a 
remedial lab do not make sufficient gains in their regular classroom. 
They have a history of being unable to achieve within one year of their 
grade level on standardized tests and sometimes are unabie to rchieve 
anything but chance scores. Because these students do not learn 
incidentally, learning opportunities must be structured to their specific 
individual needs with sequentially-based instruction such as that 
provided in the computer-based lab. For example, it can be seen using 
the pretest data that in many cases remedial students had in the past 
progressed at the rate much lower than 0.1 grade equivalent (GE) per 
month since their pretest scores were substantially below the expected 
norms in both reading and math (in the 20th percentile range). 

The review of the test indicates that the curriculum may be missing 
some highly relevant skills, often referred to as the reading/study 
skills, and these must4>e taught directly and cannot be left for students 
to attain on their own if they do not receive advanced intruction 
appropriate for secondary level students. The assumption is that all 
students need additional instruction in advanced reading/study skills. 
Students scoring in the upper range of the test norms (above 40th 
percentile) may have less to gain from the developmental reading program 
because they have higher scores to begin with. The implication here is 
that students at grade 9 need more advanced reading skills for 
independent learning. In addition, the students attending other tracks 
(1, Z, 3) should not be deprived of this opportunity but efforts should 
be considered that would provide skills in a flexibly scheduled "skills 
center 11 where students attend for. a portion of a semester (long enough to 
develop the missing skills but not necessarily of a semester-length). 
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MATHEMATICS 



When all students in grades 9 through 10 having math pretest scores 
are aggregated, the mean score obtained for the total math implies a 
somewhat higher level than for those students assigned to track c our. 
The math percentile for the overall student body is at the 27th 
percentile whereas for the track four students it is closer to the 15th 
percentile. It will be recalled that these students are generally 
performing lower to begin with. Indeed, they typically test, on the 
average, approximately 15 percentile points lower than the students 
assigned to a general /regular curriculum in the secondary school. By 
definition and pre-selection, track four contains those students who 
scored lowest on the entry level testing for numerical aoility. 

Assessment of the impact can be gained from examining the NCE scores 
in Table 3. This allows the reader to examine the results of acnievement 
tests according to whether or not they correspond with the large numbers 
of persons who took the test in the publisher's norming sample. 
Favorable results were found for these mathematics students who attended 
the computer lab and this can be readily seen by examing the gain 
column. In other words, the student growth rate *as faster than these 
same students had previously demonstrated. In this case, the students as 
a group would be considered to have performed better than expected. 
There is evidence that the support offered by this secondary school 
supplements the student's instructional program and is effective in 
accelerating the student's progress, thus compensating for his/her 
previous low achievement level. 

Total math for the students completing instruction in the math lab at 
the Greater Lawrence Technical School showed an overall gain for this 
composite of the subskills in math of seven percentile points (+7%) on 
the Stanford TASK-M. An increasing percentile rank from pretest to 
posttest is indicative of a faster growth rate than students had 
previously demonstrated. Table 3 presents the NCE data for mathematics. 
There were 98 students in the grade 9 data base assigned to 
computer-based instruction who had both a fall and spring test (paired). 
These students scored, on the average, 29 NCE on the math TASK-M pretest 
and 34 NCEs on the TASK-M posttest showing a gain of +5 NCE points. The 
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Table 3 
Students Assigned to Math 
Pretest and Posttest Results Grades 9 and 10 
Fall Grades 9-10 to Spring Grades 9-10 



Data File: GR. LAW. GRADE 9 
Paired Samples.. 

Variable: Column i Column 2 

Mean: 28.752 33.965 

StcL Deviation: 1 2.072 13.219 
Paired Observations: 98 



t-statlsttc: ■ -5.076 Hypothesis: 
Degrees of Freedom: 97 Ho: u I ■ u2 

Significance: o.OOO Ha: m » u2 



Data File: GR. LAW. GRADE 10 
Paired Samples... 

Variable: M PRE NCE n POST NC E 

Mean: 24.600 35. 1 27 

Std. Deviation: 1 2.297 1 2. 1 96 

Paired Observations: 52 



t-statistic: -6.674 Hypothesis: 

Degrees of Freedom: 51 Ho: u ! = \x2 

Significance: 0.000 Ha: \x\ * u2 
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fifty-two students in grade 10 also gained rising from an NCE of 25 to an 
NCE of 35, or an average of +10 NCE points. Students in grades 9 and 10 
in the baseline year (this first year of offering computer-based 
instruction in the developmental lab) went up and the tenth grade 
students gained dramatically. This is a higher gain than is usually 
reported for grade 10 students as many of the motivating factors of 
attending a "new school" increase scores for grade 9 students but scores 
in grade 10 tend to level off and, indeed may even go down. The fact 
that this program was able to show gains at both grades 9 and 10 is 
meaningful . 

The 150 students in the computer math lab (grades 9-10) went up an 
average of 7 NCEs as shown in Table 4. Table 4 shows NCE gain for all 
students with pre and posttest data in year 1 on the mathematics subtest 
(150 students). Even one NCE gain can be interpreted as a significant 
rise in the score. 





Table 4- 




1988 


— Stanford TASK-M 




All Students with both 


Pretest and Posttest Results - Grade 9 


Fall 






Students 




Gain 


Grade 9 N = 98 




+ 5 


Grade 10 N = 52 




+10 


N = 150 Total 




+6.7, on average 
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The analysis of math scores concluded with the students t-test to 
determine if gains proved to be significant. The analysis was performed 
using the student's t-test to evaluate statistical significance. The 
pretest to posttest statistic for grade 9 was 5.076 and for grade 10 was 
6.674 indicating statistical significance, especially for grade 10. The 
level of significance was calculated to be less than <\0005 interpreted 
as a difference that would not be obtained by chance alone in more than 
2,000 replications of the study. The t-test computed for both grades 9 
and 10 shows that the di f ferences are si gni f icant and that the math 
students do benefit from their classes for computer-based instruction. 
Math students of lower capacity visibly go up in tnis computer-based 
program and t-tests indicate a significant difference. 

The computer lab students show gains in mathematics on the NCE scale 
(average gain +7) — a gain that is more than half of the standard 
deviation of either testing (pre/post). Gains for both 9th and 10th 
grades greatly exceeded a quarter of a standard deviation unit, and thus, 
should not only be considered statistically significant, but also 
educationally significant as well. For example, the tenth grade gain of 
ten NCE points is greater than half the standard deviation of 12. Gains 
of even one quarter of a standard deviation are considered "educationally 
significant." It can be concluded that program changes made with the 
installation and implementation of computer-based instruction using 
microcomputers in the math lab impacted on the scores resulting in the 
gain of +7 NCEs for the typical student. Observable progress with 
students is recorded on the corvus (hard disc) as the students accomplish 
more of the mathematics reading objectives on the microcomputer. We can 
make the inference that the instructional program in math is maintaining 
student skills and that more skills and objectives in the school's 
curricul urn are achieved when additional supplementary instruction is 
offered through computer-based instructional lessons. 

When we look at track four students (the lowest performing) assigned 
to math classes, there is a significant gain. Students get additional 
test items correct on the posttest (compared with fall) and enough 
additional items to push them to a higher percentile. The test form 
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changes between grades 10 and 11 so that the posttest norms used for the 
sophomores are essentially based on a different testing population. 
Also, this is the year when dropout rates are higher because students 
turn sixteen. 

We can conclude that the student does benefit from his/her math 
instructional program and it can be stated that s/he: (a) gained more 
than one year of growth on the reading test; and, (b) improved his/her 
ranking among his/her peers by 7 steps (changed upward 7 places). 

Stanford Diagnostic 

For some students (117) in the mathematics program, test data from 
the Stanford Diagnostic Test were available. The Stanford is a reliable 
instrument used for measuring the magnitude of student progress 
attributable to a remedial program. The expected percentile standing of 
the students on the posttest is compared to their percentile rank on the 
pretest to determine if there is a positive sign of achievement during 
the school year; and, if there were no remedial program in place, the 
average percentile is expected to be equal in value from fall to spring. 

The Stanford provides for each student a percentile score or standard 
of relative rank based upon the student population test performance in 
relation to the norm group. The expectation of the NCE score is that, if 
everything is equal, the NCE score would not change from one time period 
to another and, if it does, the intervening program treatment nas had 
some effect. 

The Stanford Diagnostic Test yields raw scores, which are then 
converted into standard scores, percentiles and NCEs. The NCEs are then 
used to compare the participants' achievement test scores from fall to 
spring. This is the norm-referenced model used throughout this report. 
The basic assumption of the norm- referenced model is that students as a 
group will tend to maintain their relative rank or percentile standing 
over the course of a school year. 

Table 5 includes the mathematics scores for grades 9 and 10 students 
on the Stanford Diagnostic when paired fall -spring test scores were 
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available. In this case, the students as a group would be considered to 
have performed better than expected if the percentile rose from pretest 
to posttest and this is determined to be a result of the remedial program 
in the computer math lab. 

The table shows that there was a position change in -NCE scores for 
the student, on the average, for this subtest. Students rose 
approximately +9 NCE. points on the Stanford Diagnostic Test. The results 
reported in Table 5 are based upon all useable data when there was a 
valid pretest and posttest for students who remained in the program 
during the school year. There were 117 students who attained mean gain 
score of 18 NCEs on the pretest and 27 NCEs on the posttest 
administration of the Stanford. The student's t-test was computed at 
9.441 indicating a significant gain. 





Table 5 








MATH — Stanford Diagnostic 




All 


Students with both Pretest and Posttest Results 


- Grade 9-10 


Fall 










Data File: GR. LAW. GRADE 9 
Paired Samples... 

Variable: Column 7 


Column 8 






Mean: 18.321 
Std. Deviation: 1 2.058 
Paired Observations: 1 1 7 


26.465 
12.737 






t-statlstic -9.441 
Degrees of Freedom: 1 16 
Significance: 0.000 


Hypothesis: 
Ho: u 1 - u2 
Ha: Hi - u2 





Examination of the NCE scores and the percentile scores is a major 
way of analyzing the gains made by these students as a result of their 
math program. An increasing percentile rank from pretest to posttest 
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is indicative of a faster growth rate than students had previously 
demonstrated. The actual gain and the direction of the pre/post score 
difference are noted by examining these NCE gains. 

Students in this program include low-achieving students, defined as 
those in the lowest achievement quartile. This is evident from examining 
the test scores for those students selected by the criterion of low 
performance on the DAT (generally from track three or four). The 
mathematics students at the Greater Lawrence Technical School scored in 
the percentile range from 10 to 18% on the pretest. By the spring of tne 
year, after their computer mathematics lessons, students had increased 
their percentile rank (and corresponding NCE) and were scoring in the 
twentieth- to thirtieth percentile range. This pattern is similar to that 
noted for the Stanford TASK-M and thus provides further evidence to 
substantiate the claim that this computer-based instructional program is 
of great value to the students. 
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IDEAL Teacher-Made Test 



The teachers prepared a teacher-made criterion referenced test to 
match their curriculum and to be used for determining the students entry 
level when placement is provided within the computer curriculum. 

The students in grade nine and grade ten took the teacher-made 
pretest and were then entered into the placement for remedial or 
developmental instruction in the math lab. At the end of the program's 
first year, the students took the teacher-made posttest. The ninth grade 
students, on average, scored at the 51% level on the pretest and then 
scored 58% on the posttest showing a 7 point gain on the percentile scale. 

The tenth grade students scored an average (mean) of 52% on the 
pretest with a corresponding 61% on the posttest. This indicates 9% gain 
on the percentile scale. Data analysis was then completed with the 
students t-test with significant results for both grade nine and grade 
ten as shown in Table 6. 

The fact that students show a gain from pretest to posttest is highly 
significant because these are the objectives, with criterion-referenced 
test items, that the teachers deem to be educationally meaningful for 
their curriculum. Achievement of the students assigned to math, thus 
increased on three different measures, the teacher-made test, the 
Stanford Diagnostic and the Stanford-TASK-M. 

It is informative to note that the students who scored lowest on the 
DAT test, and thus thought to be of lower capacity/ability, were assigned 
to the lab for this supplementary instruction. Thus, the computer-based 
curriculum is proving to be appropriate with secondary level students 
showing these types of needs and performing in the lowest quartile of 
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their class. Students made gains on both the reading test and the math 
test when the Stanford (alternate form/level) was readministered at the 
end of the school year. These students are indeed benefitting from the 
school's program and curriculum. The achievement measure indicates that 
students are showing progress on the skills that teachers believe are 
most important for reading. 



Table 6 




Students Assigned to Math Lab 


Pretest and Posttest Results 


Grades 9 and 10 


Fall Grades 9-10 


. . to Spring Grades 9-10 


Data File* GR law gpadeq 




Paired Samples... 




\ i A P""T akin. inf A 1 l*\f>|— 

vanaoie. IDEAL PRE 


IDEAL POST 


Mean: 51.331 


57.534 


Std. Deviation: 18.799 


1 2.653 


Paired Observations: 118 




t-statistic: -3.443 


Hypothesis: 


Degrees of Freedom: 1 1 7 


Ho: u 1 = u2 


Significance: 0.001 


Ha: M 1 * U2 


Data File: GR. LAW. GRADE 10 




Paired Samples... 




Variable: IDEAL PRE 


IDEAL POST 


Mean: 51.960 


60.960 


Std. Deviation: 14.777 


13.242 


Paired Observations: 75 




t-statfstic: -5771 


Hypothesis: 


Degrees of Freedom: 74 


Ho: ul =M2 


Significance: 0.000 


Ha: u I * \i2 
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Shift in Rank Placement 



One criterion of a successful reading or math program is its ability 
to move students out of lower quartiles in comparison to the national 
norms on the SRA Test. Data were examined to review how many students 
actually shift this rank placement; the shift into another quartile 
(e.g., from the 24th percentile or 1st quartile to above the 25th 
percentile or into the 2nd quartile) is significant growth at the 
secondary level in reading or math. Movement out of the 1st quartile 
indicates that the lower performing students on the pretests move up in 
rank in comparison to national norms for the Stanford TASK. 

Descriptive statistics on the tests showed that the students are 
changing relative to rank placement from fall to spring. A noticeable 
positive shift of the distribution of students is apparent in the data. 
Some students maintain their rank in their group/class (because in 
essence the standard is raised for 10th grade). The fact that students 
are doing as well or better on the posttest (better than expected in 
contrast with the national norms) indicates that pupils who met the 
objectives of the school's instructional program oenefitted from the 
individualized attention provided in the program and the school -wide goal 
of basic skills. Those students in the first quartile shifting into the 
second quartile accounted for a smaller percentage who made gains; 
whereas some students shifting from the second quartile into the third 
quartile showed exceptional achievement. 

In the grade 9 math data base, as many as 17 students moved ahead in 
a quartile shift; this represents 17% of the matched set of students with 
data for both pre and posttesting. On the Stanford test, about 31% of 
the students in grades 9-10 make a quartile shift. In grade 10 students 
moved ahead in rank placement representing 27% of the data file. 

Differential Assignments 

Whe^ some students were assigned to math and some were assigned to 
reading, the NCE gains were as shown in Table 5. The students assigned 
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to the reading program (N = 24) are making a gain of +6 NCEs (from a mean 
of 26 to a mean of 32 NCEs). The students assigned to the math program 
are making a gain of +7 NCEs on the average. These data would seem to 
indicate that the computer-based instruction that was instituted helps to 
raise the reading comprehension and the mathematical skills for the 
students enrolled. Thus, something different has occurred to help 
increase student scores and it can be inferred that it was the 
instructional program using computers for skills practice and application. 
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SUMMARY 



Growth in achievement from the fall of 1987 to the spring of 1988 has 
been reviewed by the evaluators. The achievement for the students was, 
by definition of the program selection criteria, below average on the 
pretesting during the fall of 1987 (i.e., below the 40th percentile). 

The purpose of the analysis of the achievement data is to deterinins 
whether the treatment produced achievement gains in excess of those 
observed in the sample when the test was normed. This NCE model (known 
as Model A) compares students who received instruction to the test norms 
for the national norming population. The basic assumption of the 
norm-referenced model is that students as a group will tend to maintain 
their relative rank or percentile standing over the course of a school 
year. If the average relative performance of the students on the spring 
testing is better than the average relative performance on the fall 
testing, then the program is determined to have a positive impact on 
student achievement. 

Whenever the evaluation shows an NCE gain greater than zero, it means 
that the students profited from participating in the project. In 
general, the larger the NCE gain, the more efffective the remedial 
instruction. Because large numbers of student scores are needed to 
actually document that a gain is occurring (thirty students or more) the 
reading scores are to be read with caution and larger groups of students 
will be needed to generalize for the ninth grade program. 

NCE scores are used for this evaluation to look across the grades 
9-10 in the mathematics program to determine overall impact of- the 
program. When examining the overall change in scores for all grades 
(9-10) a gain score, on the average, of +7 NCEs across all students is 
quite respectable and is educationally as well as statistically 
significant. Typically, a school might expect that students would gain 5 
to 7 NCEs in the elementary program; to gain this much on the MCE scale 
in the secondary school is particularly meaningful. 
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Scores on the NCE scale were significantly higher for the spring 
administration of the Stanford Tests. This can be readily seen by 
examining the pre-post gains on each table showing a +5 NCE gain in 
reading and an average gain of +7 NCE in math. This represents 
outstanding progress. Pre-pos*ttest scores were available for 130 
students served in mathematics who took the TASK-M but only twenty-four 
students in reading who took the TASK-R. In addition, test scores 
(paired) were available for 117 students on the Stanford Diagnostic Math 
Test who showed an average gain of +9 NCEs. 

The gains in the Greater Lawrence program are substantially above the 
expected progress for students and are highly commendable, especially in 
this their first year of the computer-based instruction in the lab* 
Other factors are brought into the Inferences for this project and for 
students served and these include test taking skills, the fact that ninth 
graders are entering a school that is "new" to them as well as being 
vocational in orientation, and the fact that 10th grade students have 
been in the program for two years, attitudes of the students and the 
teacher-student relationship, as well as the relationship of the 
curriculum in the computer lessons to the test. 

It is recommended that the staff examine these remaining factors and 
begin to get a handle on the variables as they pursue the depth of this 
quality instruction that is being offered to their secondary students* 
In particular, the selection and administration of ample testing, to 
measure all of the objectives for this curriculum, would be recommended. 
In the 1988-1989 school year, reading teachers have been assigned to the 
reading lab instructional period to work with the lowest performing 
students. This will prove to be a valued asset to the program. 

There is evidence of direct student gains in both reading and 
mathematics attributable to the provision of this computer-based 
instructional program. Substantial gains wer-e achieved in mathematics on 
the Stanford Diagnostic Test and the Stanford TASK-M. In other words, 
the student growth rate was faster than these same students had 
previously demonstrated. Student gains offer evidence that the 
instructional support provided by the teachers in the computer laboratory 
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successfully impacted achievement scores. The positive findings reported 
in these achievement gains were substantiated by teachers and the program 
director noting that students had made significant progress. 

When gain scores can be analyzed for over 130 students in a grade 
span of two grades, which is the case in the math data presented, the 
number of students increases making the score reported one that is highly 
reliable. Therefore, when standardized tests can be used, the "tighter" 
the evaluation design. Based on the testing data, student progress, and 
on-site observations, it can be concluded that the Greater Lawrence 
program made a significant impact on the level of student achievement 
during the 1987-1988 school year. The test results summarized here 
demonstrate that the program was highly successful. Based on the 
findings derived from this report, it can be concluded that the program 
made an excellent impact on the level of student achievement. The 
analyses carried out and reported were intended to test whether the 
program produced achievement gains and the results support this 
conclusion. The Greater Lawrence program is an exceptionally fine 
instructional program and the staff can feel assured that their efforts 
are well worthwhile. The director and the faculty can be pleased with 
the success of this program. 

Conclusions based on this study indicate that instructional 
microcomputing using a system such as IDEAL 1 s computer-based 
instructional components can be a valuable educational tool. 
Additionally, it is believed that affective factors, such as motivation 
and self-esteem, are enhanced and this makes for a prevalent concept of 
success at the secondary level where traditionally morale is often low 
because of student's assigned to "track four" not showing repeated 
attempts at progress but sinking back into the mire of despair. In the 
computer-based instructional component, more applications and 
concentrated practice can be regarded as essential and the time and 
intensity of instruction, when increased, should continue to show 
advancing gains. An option at this time is to add an additional test 
that would serve diagnostic-prescriptive purposes in addition to the 
teacher-made test. This test should have separate subtests for concepts, 
applications, and numerical confutation. There is a need to seek out 
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more answers to this question of how the instructional program has a 
differential impact on students of varying capacities, abilities and 
interests. The students of lower capacity are assigned to the reading 
program and math program* 

Installation in Greater Lawrence was important to initiate early 
summation in order to gain an indication as to whether the installation 
was in fact making a difference for students. To accomplish this, the 
Massachusetts Department of Education Division of Occupational Education 
provided a small grant ($5,000) to examine data in a pre-post basis to 
determine if any gains were achieved in the first year. What is not 
covered in this study are any kinds of formative evaluation data 
examining elements of scheduling, organizing, time on task, and 
longitudinal study information using control groups. Results from this 
study make a case for now suggesting that a more comprehensive study 
follow this preliminary intervention project to emphasize scores for 
students in vocational education. 

The first year of this math program for the students of lower 
capacity demonstrates evidence of a positive effect. Continuing reading 
and math into a second year will prove most valuable. Certainly, the 
Stanford test gets harder and it takes more correct answers, on more 
sophisticated items, to get the same percentile score in later grades. 
By continuing to adapt the curriculum at 9th grade it would be possible 
to continue to show growth on the Stanford-TASK. The stated basic 
skills test alone is inadequate to measure the sophisticated 
reading/study skills and mathematics applications. Continued use of the 
Stanford TASK and the Stanford Diagnostic Tests is recommended to 
continue monitoring this program. 

The teachers reported favorably on the use of the computer as a 
motivating learning tool. The computer also provides an instructional 
management system that links the diagnostic items and teaching modules 
for use by the high school mathematics and reading laboratory so that 
each student is working with appropriate lessons. Teachers presented a 
diagnostic math te r t at the beginning of the year to ascertain entry 
levels for each student in the mathematics curriculum. Two emerging 
forces can help to provide a sound basis for the introduction of these 
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teaching tools into the secondary curriculum: (a) inexpensive, 
cost-efficient processing power to provide the necessary instructional 
information for decision-making; and, (b) growing understanding of the 
cognitive tasks required in the curriculum. This program at the 
secondary level has further provided evidence to encourage course 
development and demonstration. The computer-based instructional 
component assists teachers in reexamining the basic skills curriculum in 
light of changes brought about by the introduction of these new tools. 

The inclusion of microcomputers in an instructional lab setting at 
the secondary level and the use of instructional microcomputing is 
extremely effective as an adjunct to the instructional program and 
methods of the secondary school. Instructional computering using a 
computer-based system for mathematics and reading are demonstrated here 
to be valuable educational tools and most effective as an approach to the 
secondary curriculum. 
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MATH 6 



1. ) Write the numerals for the following: Don 't forget Commas . 

Three Hundred forty-seven thcu.* .nd, twenty 

a. 347,020 c. 347,000,020 

b. 347,200 d. 347,000,200 

2. ) 

Round 47,268 to the nearest thousands place. 

a. 40,000 c. 47,300 

b. 50,000 d. 47,000 

3. ) Find the suit. 

235 + 422 * 

a. 576 c. 756 

b. 657 d. 567 



e. None of the Above 



e. None of the Above 



e. None of the Above 



4.) Find the sum. 

i 

4,593 
5,784 
1,629 



a. 101,896 

b. 10,006 



c. 21,006 

d. 60,021 



e. 12,006 



5.) Find the difference for each of the following problems. 
349 - 24 » 

a. 25 c. 365 e. None of the Above 

b. 373 d. 325 
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6.) 7,414 

a. 7,184 

b. 7,016 



398 = 



c. 7,812 

d. 7,114 



e. None of the Above 
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MATH 6 



(continued) 



7.) Multiply 

568 
x 7 



8.) 625 
x37 



9.) Divide 
7fZ4T~ 



a. 1,275 

b. 3,646 



c. 3,985 

d. 3,976 



a. 23,125 

b. 672 



C. 22,125 
d. 24,225 



10.) 9312b, adV 



a. 35 

b. 30 

a. 275 R 82 

b. 285, ft 72 



c. 138 

d. 5 

c. 287 R 62 

d. 285 R 82 



11. ) Which of these numbers are divisible by 3? 
a. 29 b. 141 c. 271 d. None of these 

12. ) Name the next multiple of 5 in this series: 

115 , 120 , 125 
a. 120 b. 50 



c. 130 



d. 75 



13.) Which number is divisible by 9? 
a. 14 b. 22 c. 19 



e. None of the Above 



e. None of the Above 



e. None of the Above 



e. None of the Above 



d. 54 



e. None of these 



14.) Which number is a multiple of 10? 

a. 125 b. 130 c. 142 d. None of these 



15.) Which of these numbers is prime? 
a. 72 b. 23 c. 9 
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3. None of thsse 
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MATH 6 

(continued) 



16. ) Which of these numbers is composite? 

c< «, d . 31 e. None of these 

17. ) What are the common factors of 16 and 24? 

a. 1. 2, 4. 8 c. 1, 8, 16, 24 

b. 1, 4, 16 <*• 2, 4, 6 

18. ) Find the next multiple in the ser-s for the factors 3 and 5 ? 

15, 30, -2- 

a. 60 c. 10 

b. 45 d - 25 

19. ) What fraction is equivalent to %1 

a. 3 b. 3 c. 3j d. 3 2 



20.) 4 _ 16 



a. 10 c. 20 



5 " r b. 15 d. 25 

21. ) 75 a. 25 c. 30 

r- = 

3 b . 15 d. None of the Above 

22. ) 5 ? 7 a- < b « > c * = 

5 9 

23. ) Simplify: 18 a. 6^ b. 5^ c. 2^ 

i~5 15 9 5 

d. None of the Above 



24.) Add + Simplify if possible 

3 § a.4 b.^ c. 3, d.4 

iO 



♦ io " ^ 33 15 



ERIC 5j 



MATH 6 

(continued) 



25.) Add + Simplify if possible: 



1 

6 



+ 2 

18 



a. 6 
T8 



b. 2 
3 



c. 9 

18 



26.) Find the difference, simplify if possible: 



5 
5 



4 
6 



a. 1 
5 



b. 2 
6 



c. 1 



d. 1 
3 



None of the Above 



27.) 8 

g 



4 

5 



a. 1 

g 



b. 4 
$5 



c. 3 
53 



d. 1 
TS 



28.) Express these fractions as mixed numerals in lowest terms: 



54 
TO 



a. c 1 
6 5 



c. c2 
5 5 



d. c 4 
5 TO 



29.) Add and Simplify if possible: 
1 . c 3 



5 3- + 6 5 

f 



11 3 5 



b. 12 1 
12 2 



c n 4 



30.) Find the difference and simplify if possible: 



8 



b. 



c. 12 



d. 14 
11 T5 



d. c 5 
5 6 



31.) 113 



78 



18 
23 



a. 34 1 



TO 



b. 



34 



20 



35 



1 d. „2 



TO 



35 



20 



32.) 9 1* - 6 % 



a. 2k 



b. 3% 



c. 3'A 



d. 2 3/4 



33.) 3 2 
5 x f 



a. 6 
T5 



b. 5 
3 



c. 5 
TS 



d. 2 
5 



34.) 2 
3 



x 14 



a. 9 1 b. 28 c. .8 d. None of these 
3 3 



6 1 4 - 



a. 9 
26- 
15 



b. 



27 h 

c ~» 

«-"0 



«• 26 3. 



D.. None of These 
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6 

(continued) 



36. ) What is the Reciprocal of : 6 

a. 1 b. 6 c. 61 d. None of these 
6 ' 

37. ) Divide, then Simplify if possible. 

4.2 a. 4 b. 26 c. 1 d. 12 

I < j 4 1- M 

38. ) 4.1 a. 1 b. 32 c. 4 d. 1 

T 3 32 8 2 

39. ) , 1 ,1 a. ,1 b. 9 c. 49 d. None of these 

6 3 T \ h 5 a - 



40. ) Write the decimal numeral for the following numbers. 

forty - five and two hundred sixteen thousandths. 

a. 45,216 c. 45.00216 

b. 45.216 d. 45 216 

TOGO 

41. ) Write >, <, or = in the blank. 

5.56 5.568 a. > b. < c. * d. None of the above 

42. ) Find the sum or difference for each of the following problems. 

95.3 a. 10.007. c. 101.7 e. None of the Above 

+6.4 

T — b. 100.07 d. 10.007 

43. ) 96.8 a. 52.2 c. 141.4 

- 44.6 e. None of the Above 

b. 14.14 d. 5.22 

44. ) Find the product for each of the following decimal problems. 

.96 x 58.1 a. 55.776 c. 5.5776 e. None of the Above 

b. 67.7 d. 557.76 
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MATH 6 

(continued) 



45. ) Find the quotient for each decimal problem. Be sure to place the decimal point 

properly into quotient. 

15|Ou~" 

a. .30 

b. 3.0 d. 03 

c. 30 e. None of the Above 

46. ) Find the quotient for each decimal problem. Be sure to place the decimal 

point properly into quotient. 

.0617^ 

a. 3 

b. .03 d. -002 

c. .02 e. None of the Above 



47. ) .41 | 25.584 

a. 5.113 c. 51.1 

e. None of the Above 

b. 62.4 d. .624 

48. ) Write a fraction for the following decimal numbers. 

.6 

a. 6 c. 6 e. None of the Above 
TO TOO 

b. 3 d. 6 
I T 

49. ) Change the fractions to decimal numbers. If the decimal is repeating, show 

this with the proper symbol. 

3 

TT 

a - 1 c ' 273 

e. None of the Abc\e 



MATH 6 

( continued) 

50. ) Change the fraction to a decimal number and round to the nearest hundredth. 

6 
7 

a. .75 b. .86 c. .85 d. .851 e. None of the Above 

51. ) Change the fraction to a decimal number, and round to the nearest thousandth. 

5 
Tl 

a. .455 b. 2.2 c, .46 d. .454 e. None of the Above 
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